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1.  INTRODUCTION 


This  is  the  second  semiannual  report  on  the  research  project  entitled  "Automated  Handling 
and  Assembling  of  Non-rigid  Objects",  covering  the  period  July  15, 1990  through  January  14, 
1991.' In  this  period,  three  topics  were  studied.  The  first  topic  addresses  the  strategies  of 
automated  handling  and  assembling  of  non-rigid  objects  by  using  multiple  end-effectors,  the 
second  topic  studies  the  shape  reconstruction  of  two  dimensional  objects,  and  the  third  topic  is 
the  construction  of  a  laser  iange  finder  for  identification  of  the  third  dimension  of  an  object. 

Y  The  purpose  ef  the  first  topic  is-tj  investigate3 optimal  tool  structures  for  handling  one  and 

•  •  •  ' J  *  i  f  t  '  ' ' ' 

two  dimensional  objects.  Previously, ywe  proposed  to  us®* vacuum  pads  to  pick  up  large 
deformable  sheet  by  the  difference  of  the  air  pressure.  Such  a  mechanism  needs  a  large  pad 
when  the  object  has  a  large  surface  area,  which  is  often  too  heavy  for  numerically  controlled 
machine  to  carry.  By  using  two  ordinary  end-effectors,  just  like  the  two  hands,  of  human 
beings,  deformable  objects  can  be  manipulated  without  using  a  large  tool.  'Our  study  has 
developed  an  optimal  mechanism  for  coordinating  the  end-effectors  such  that  the  machine 
consumes  the  minimum  energy  and  no  damage  is  made  to  the  object. 

■  The  purpose  of  the  second  topic  is4e>  identify,  the  deformed  shape  of  two-dimensional 
objects.  For  two  dimensional  objects,  there  does  not  exist  closed-form  solution  to  the 
governing  equation  of  the  deformation.  Our  goal  is  to  sense  a  few  data  points  on  the  deformed 
surface.  Based  on  these  points,  a  numerical  model  can  be  driven  which  can  reconstruct  the 
entire  surface  with  a  good  approximation.  This  numerical  model  can  also  be  modified  in 
accordance  with  the  deformation  behavior  of  the  object. 

,The  purpose-ef  the  third  topic  ifr-te,  develop  a  laser  range  finder  such  that  all  three 
dimensions  of  an  object  can  be  identified.  The  laser  range  finder  is  a  very  important  tool  for 
our  research,  since  a  two-dimensional  object  becomes  three  dimensional  once  it  is  deformed. 
For  three  dimensional  objects,  the  third  dimension  is  important  to  completely  identify  the  shape 

of  an  object  even  before  the  deformation  occurs.  . . - 

The  rest  of  this  report  describes  the  results  or  the  status  of  the  three  topics  as  just 
summarized  as  well  as  the  research  plan  for  the  next  period. 
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2 .  AUTOMATED  HANDLING  OF  DEFORMABLE  OBJECTS  BY  COORDINATED 

MANIPULATORS 

Traditionally,  large  two-dimensional  objects  are  handled  by  vacuum  pads.  The  pads  are 
designed  to  cover  the  most  area  of  the  object  surface.  When  the  space  inside  the  pad  becomes 
vacuum,  the  material  can  be  picked  up  by  the  difference  of  the  air  pressure.  A  vacuum  pad, 
however,  may  become  very  heavy  and  big  when  the  object  surface  is  large.  This  imposes  a 
large  load  on  the  machines  on  which  the  pad  is  installed.  Human  beings,  on  the  other  hand, 
can  effectively  handle  large  deformable  objects  with  two  hands.  This  implies  that  by 
coordinating  two  manipulators,  the  same  object  may  be  handled  by  ordinary  end-effectors. 
The  mechanism  for  handling  deformable  objects,  however,  is  very  different  from  handling 
rigid  objects.  A  major  difference  is  that  the  two  end-effectors  have  certain  degrees  of  freedom 
in  their  relative  motions.  This  motion  freedom  can  be  used  to  satisfy  other  requirements  in  the 
automated  handling  of  the  objects.  For  example,  reducing  the  distance  between  the  two  end- 
effectors  can  reduce  the  workspace  required  by  the  automation  process.  The  question  is  what 
are  the  optimal  motion  trajectories  for  each  end-effector  in  handling  the  objects? 

We  study  this  problem  by  using  two  criterion.  First,  the  load  imposed  to  the  handling 
machine  should  be  as  small  as  possible,  and  second,  the  possibility  of  damage  to  the  object 
should  also  be  as  small  as  possible.  Since  the  flexibility  of  the  object,  the  relative  position  and 
orientation  of  the  two  end-effectors  can  be  altered  while  handling  the  object.  When  the  object 
is  deformed,  however,  deformation  forces  and  moments  are  exerted  on  the  end-effectors  and 
the  objects.  Large  deformation  forces  and  moments  may  damage  the  object  and  impose  a  large 
load  on  the  handling  machine. 

Three  coordinating  methods  are  suggested  for  the  two  manipulators.  The  first  method 
allows  the  relative  orientation  between  the  end-effectors  to  be  altered,  but  no  relative  position 
changes.  The  second  method  allows  the  relative  position  to  be  altered,  but  no  relative 
orientation  changes.  The  third  method  allow*  both  the  position  and  orientation  to  be  changed. 
Through  the  analysis  of  kinematic  constraints,  the  first  method  is  found  not  possible,  Studies 
are  then  concentrated  on  the  second  and  third  methods.  The  second  and  third  methods  arc 
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further  compared  through  the  study  of  deformation  mechanics.  It  is  shown  that  the  third 
method  results  in  minimum  reaction  forces  and  moments  between  the  objects  and  end- 
effectors.  It  is  identified  as  the  best  method  for  automated  handling  of  two  dimensional  and 
deformable  objects. 

Unfortunately,  motion  trajectories  for  the  two  manipulators  to  handle  the  object  are  very 
complicated  to  compute,  which  is  not  adequate  for  real-time  execution.  For  practical 
applications,  we  further  propose  a  simplified  method  in  which  the  complicated  trajectories  are 
replaced  by  piece-wise  linear  functions.  The  reaction  forces  and  moments  in  the  simplified 
method  are  also  analyzed  which  are  only  slightly  larger  than  the  original  method. 

To  veiify  the  proposed  method,  experiments  are  also  conducted  in  our  Advanced 
Manufacturing  Laboratory  .  The  experiment  results  show  that  the  third  method  (Fig.  1)  is 
better  than  the  second  method.  By  using  the  second  method,  the  orientation  of  the  two  end- 
effectors  are  kept  unchanged  while  they  approach  each  other  in  order  to  bend  the  object.  This 
exerts  large  reaction  forces  and  moments  on  the  objects,  leaving  two  permanent  crease  on  the 
object  surface.  By  using  the  third  method,  the  end-effector  simultaneously  alter  their  position 
and  orientation  while  bending  the  object,  and  no  permanent  damage  is  made  on  the  object.  In 
addition,  the  forces  and  moments  exerted  on  the  end-effectors  are  much  smaller  than  that  in  the 
second  method. 

A  technical  paper  has  been  written  based  on  the  research  results  just  described,  which  is 
attached  as  Appendix  A  of  this  report.  Technical  details  of  the  handling  mechanism  by  two 
manipulators  can  be  found  in  the  paper. 

3.  SHAPE  RECONSTRUCTION  OF  TWO  DIMENSIONAL  OBJECTS 

Shape  reconstruction  of  two-dimensional  objects  is  a  special  problem  associated  with  the 
automated  handling  and  assembling  of  deformable  objects.  When  a  two-dimensional  object  is 
picked  up  by  an  end-effector,  the  object  is  deformed.  In  order  to  precisely  position  it,  the 
deformed  shape  needs  to  be  identified.  There  are  two  difficulties  associated  with  this 
identification  task.  First,  the  deformation  characteristics  of  the  object  are  unknown;  therefore, 
it  is  impossible  to  reconstruct  the  surface  by  using  its  deformation  governing  equations. 
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The  principle  of  physically-based  modeling  for  vision  can  be  stated  as  follows:  physical 
properties  and/or  high  level  knowledge  of  the  objects  should  be  considered  in  both  their 
representation  and  die  low  level  sensing  tasks  of  the  vision  sensor.  Thus  the  shape  can  be 
reconstructed  by  using  sparse  data  points  of  the  surface  as  the  rest  of  the  surface  should  be 
governed  by  the  physical  properties  of  the  object.  If  this  principle  is  applied  to  shape 
reconstruction  of  a  two-dimensional  object,  the  deformation  governing  equations  of  the  plate 
should  be  used  to  derive  the  representation  of  the  objects.  However,  as  mentioned  earlier,  this 
is  not  possible.  Instead  of  strictly  following  the  deformation  governing  equations,  we  propose 
a  physical  constraint  concept,  i.e.,  physical  constraints  associated  with  a  deformable  object 
should  be  considered  when  the  shape  is  constructed.  The  physical  constraints  include  the 
elastic  deformation  of  two-dimensional  object,  the  shape  of  the  object  before  deformation,  and 
the  position  and  orientation  of  the  gripper  that  holds  the  object.  These  constraints  are  already 
known  and  should  be  taken  into  consideration  in  the  process  of  reconstruction. 

In  order  to  reflect  the  elastic  nature  of  the  object,  the  shape  of  the  object  is  proposed  to  be 
reconstructed  by  minimizing  the  strain  energy  of  the  object  surface.  As  the  expression  of  the 
plate's  strain  energy  is  very  complex,  it  is  simplified  for  the  purpose  of  efficient  computation. 
This  simplified  strain  energy  happens  to  be  the  same  as  the  energy  functional  of  the  thin  plate 
spline  (TPS)  used  in  visual  surface  reconstruction.  A  pioneering  work  in  visual  surface 
reconstruction  is  accomplished  by  Grimson  [1].  Terzopculos  [2]  gave  a  more  rigorous  proof 
of  the  TPS  in  reconstructing  the  surface  from  given  sparse  data  point.  Both  Grimson  and 
Terzopoulos  used  energy  minimization  methods  to  reconstruct  the  surface.  Their  method  is 
adopted  in  our  reconstruction  process. 

In  the  work  of  visual  reconstruction  done  by  Grimson  and  Terzopoulos,  the  sparse  set  of 
data  points  is  provided  by  stereo  visions.  We  propose  to  use  a  laser  range  finder  in  our  shape 
reconstruction  process.  The  advantage  of  using  the  range  finder  is  that  we  can  control  the 
sensor  to  sense  the  data  points  we  want.  The  range  finder  can  be  moved  by  a  robot  arm. 
Hence,  the  sensor  is  dynamic.  We  thus  have  to  decide  sensing  strategies  for  the  range  finder. 

The  basic  strategy  is  that  we  need  to  sense  dense  data  points  from  the  region  where  the  the 
surface  curvatures  are  large  and  sparse  data  points  from  the  area  where  the  surface  curvatures 


are  small.  The  physical  constraints  as  mentioned  earlier  can  determine  whether  the  curvatures 
on  a  particular  part  of  the  surface  arc  large  or  small. 

At  the  point  of  this  writing,  this  reconstruction  topic  is  still  under  active  investigation.  We 
expect  that  a  detailed  technical  report  will  be  available  when  we  submit  next  semiannual  report. 

4.  THE  CONSTRUCTION  OF  A  LASER  RANGE  FINDER 

In  order  to  carry  on  experimental  study  in  our  research,  a  laser  range  finder  was 
constructed  in  this  research  period.  The  basic  structure  of  the  laser  range  finder  is  as  shown  in 
Fig.  2.  It  consists  of  a  laser  and  a  CCD  camera.  The  laser  generates  a  beam  light  source  which 
is  converted  into  a  sheet  of  light  through  a  cylindrical  lens.  This  sheet  of  light  is  scanned 
across  the  scene,  producing  a  single  light  stripe  for  each  position.  When  the  light  stripe  is 
sensed  by  the  CCD  camera,  the  camera  view  of  the  stripe  shows  displacements  along  a  stripe 
which  are  proportional  to  depth.  Thus,  a  structured  light  source  plus  a  two-dimensional  sensor 
can  provide  three-dimensional  information.  Three-dimensional  information  is  essential  for  us 
to  study  the  deformation  behavior  of  two  and  three-dimensional  objects. 

At  the  point  of  this  writing,  the  laser  range  finder  is  already  operational.  Appendix  B  of 
this  report  describes  how  to  locate  a  point  and  a  surface  in  a  three-dimensional  space  knowing 
only  information  about  their  image  captured  by  the  range  finder. 

5.  RESEARCH  PLAN  FOR  THE  NEXT  SEMIANNUAL  PERIOD 

In  the  next  period,  we  plan  to  continue  our  research  on  the  reconstruction  of  the  two- 
dimensional  object  as  described  in  the  third  section  of  this  report  In  addition,  we  will  start  to 
look  at  the  virtual  space  issue  as  mentioned  in  our  proposal.  This  issue  is  for  planning  the 
motion  control  mechanism  of  the  host  machines  which  handle  the  object.  The  basic  idea  is  to 
use  a  virtual  space  to  enclose  the  shape  of  a  deformable  object  As  a  result  the  deformed  shape 
of  an  object  does  not  need  to  be  exactly  identified  as  long  as  it  falls  in  the  virtual  space.  In  this 
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Fig.  2  'Hie  structure  of  the  range  finder 

way,  we  can  plan  the  motion  trajectory  of  the  host  machine  based  on  the  virtual  space,  and  can 
still  avoid  collisions  of  the  object  with  any  other  obstacles. 

Another  topic  that  we  will  initiate  is  the  deformation  of  three-dimensional  objects.  We  will 
first  study  how  a  three-dimensional  object  is  deformed  under  external  forces  and  moments. 
Then,  we  will  investigate  how  to  recognize  a  three-dimensional  object  when  it  is  deformed. 
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AUTOMATED  HANDLING  OF  DEFORMABLE  OBJECTS  BY  COORDINATED 

MANIPULATORS 

Ming  Z.  Chen  and  Yuan  F.  Zheng 
Department  of  Electrical  Engineering 
The  Ohio  State  University 
Columbus,  OH  43210 

ABSTRACT 

Coordinating  two  manipulators  to  automatically  handle  deformable  objects  is  studied  in 
this  paper.  Because  of  the  flexibility  of  the  objects,  the  manipulator  hands  are  allowed  to  have 
certain  degrees  of  freedom  in  their  relative  motions.  However,  kinematic  constraints  are  still 
imposed  on  the  relative  motions  between  the  two  manipulators.  In  this  study,  the  kinematic 
constraints  are  first  derived  from  the  flexibility  of  the  objects.  Then  the  investigation  is 
concentrated  on  the  mechanisms  for  handling  one-dimensional  objects.  Three  coordinating 
methods  are  identified  for  the  manipulators.  The  first  method  allows  the  relative  orientation 
between  the  two  manipulator  end-effectors  to  be  altered,  but  no  relative  position  changes.  The 
second  method  allows  the  relative  position  to  be  alterec,  but  no  relative  orientation  changes. 
The  third  method  allows  both  the  position  and  the  orientation  to  be  altered.  Through  the 
analysis  of  the  object  mechanics,  it  is  found  that  the  first  method  is  not  adequate.  Motion 
trajectories  of  the  end-effectors  are  then  derived  for  the  second  and  third  methods.  Analysis 
also  shows  that  the  third  method  results  in  the  minimum  reaction  forces  and  moments  between 
the  object  and  the  hands.  It  is  best  to  use  for  automated  handling  of  one  or  two  dimensional 
deformable  objects.  Finally,  experimental  results  are  presented  to  verify  the  theoretical  studies. 
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1.  INTRODUCTION 


In  recent  years,  coordinating  two  or  more  robot  manipulators  for  manufacturing 
automation  has  received  much  research  attention.  An  earner  investigation  was  conducted  by 
Konstantinov  and  Markov  [1].  In  their  work,  some  necessary  conditions  for  the  collaboration 
of  two  end-effectors  in  assembly  operations  were  given.  Later,  multiple  manipulators  working 
in  a  common  workspace  were  studied  by  Freund  [2].  His  main  interest  was  to  avoid 
collisions  among  multiple  manipulators,  for  which  a  hierarchical  control  system  was 
developed,  and  a  nonlinear  control  mechanism  was  used  in  the  system. 

Theoretical  studies  of  kinematic  constra  aits  imposed  on  two  coordinating  robots  handling 
a  rigid  object  were  conducted  by  Mason  [3].  Zheng  and  Luh  later  extended  Mason’s  results 
by  identifying  one  manipulator  as  the  leader,  and  the  other  as  the  follower  [4,5].  Kinematic 
constraints  were  also  formulated  in  [4,5]  for  the  leader  and  follower  when  they  handled  rigid 
objects  as  well  as  objects  with  degrees  of  freedom,  such  as  a  pair  of  pliers.  Hemami  [6],  on 
the  other  hand,  gave  another  approach  to  study  the  coordination  control  of  two  moving  arms, 
in  which  the  rigid  body  constraint  equation  was  replaced  by  a  symmetry  rela^onship  between 
the  arms.  More  recently,  Lee  [7]  extended  the  concept  of  the  robot  manipulability  to  redundant 
dual-arm  systems.  He  pointed  out  that  the  required  motion  and  force  trajectories  of  a  given  task 
by  a  redundant  dual-arm  system  could  be  abstracted  by  a  series  of  desired  manipulability 
ellipsoids.  Furthermore,  task-oriented  dual-arm  manipulability  could  be  mathematically 
defined  by  quantifying  how  the  manipulability  of  one  arm  affects  the  other  and  measuring  the 
geometrical  close  •'.ess  between  the  desired  and  the  actual  manipulability  ellipsoids. 

Dynamics  of  two  coordinating  robots  have  also  been  extensively  studied.  In  [8,9],  it  was 
revealed  that  two  dynamic  equations,  one  for  each  coordinating  robot,  could  be  combined  to 
form  one  unified  dynamic  equation  when  the  two  robots  were  handling  a  rigid  object.  In  the 
control  aspects  of  two  coordinating  robots,  Tam,  Bejczy  and  Yun  [10,11]  developed  a 
nonlinear  dynamic  control  method.  The  main  feature  of  their  method  is  that  the  nonlinear 
feedback  mechanism  was  integrated  with  an  optimal  error  correcting  loop  and  an  optimal 
coordinator.  Ozguner,  Yurkovich  and  Al-Abbass  [12]  established  a  hierarchical  framework 
which  employed  two  levels  of  control  hierarchy.  The  decentralized  model  reference  adaptive 
control  approach  using  variable  structure  controller  was  applied.  Hayati  [13],  and  Yoshikawa 
and  Zheng  [14]  extended  the  theory  of  hybrid  position/force  control  to  the  case  of  multiple 
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coordinated  robots.  In  [13],  the  solution  was  given  by  partitioning  the  object  and  considering 
the  object  as  part  of  the  last  link  of  each  arm,  and  cooperation  was  achieved  by  controlling 
each  arm  such  that  the  bm^en  of  actuation  was  shared  between  the  arms  in  a  non-conflicting 
way.  The  method  proposed  in  [14]  could  control  the  motion  of  the  object  as  well  as  the 
forces  exerted  on  the  object.  In  another  work,  Koivo  [15]  proposed  that  the  two  coordinated 
robots  could  be  controlled  by  an  adaptive  controller  of  a  self -tuning  type.  The  controller  was 
designed  on  the  basis  of  the  stochastic  multi-variable  discrete  time  model,  in  which  the  system 
parameters  were  recursively  estimated  on-line  by  using  a  quadratic  criterion. 

While  the  studies  on  the  coordination  of  multiple  manipulators  continue  [16-20],  one 
aspect  has  never  received  serious  attention,  i.e.,  handling  deformable  objects  by  multiple 
manipulators  In  reality,  however,  there  is  a  frequent  occurrence  of  non-rigid  objects.  For 
instance,  in  the  shipbuilding,  aerospace,  and  automobile  industries,  flexible  plates  have  long 
been  used  in  the  assembly  of  various  kinds  of  vehicles.  It  is  more  noticeable  in  recent  years 
that  non-rigid  composite  materials  such  as  pre-preg  have  been  used  to  replace  metal  in  many 
places  [21],  Traditionally,  to  handle  non-rigid  materials,  special  toolings  needed  to  be 
designed  such  as  a  vacuum  pad  [21].  The  pads  are  designed  to  cover  the  most  area  of  the 
object  surface.  When  the  space  inside  the  pad  becomes  vacuum,  the  material  can  be  picked  up 
by  thv  difference  of  the  air  pressure.  However  such  a  mechanism  needs  a  large  pad  when  the 
object  has  a  large  surface  area,  which  often  becomes  too  large  and  too  heavy  to  be  carried  by  a 
robot  manipulator.  Human  beings,  on  the  other  hand,  can  effectively  handle  large  deft  mable 
objects  with  two  hands.  This  reminds  us  that  by  coordinating  two  or  more  manipulators,  it  is 
possible  to  handle  deformable  objects  by  using  ordinary  end-effectors.  As  a  result,  large  and 
heavy  tools  can  be  avoided.  This,  however,  raises  a  new  technical  problem  on  coordination, 
i.e.,  hew  to  coordinate  motion  trajectories  of  the  two  manupulators  such  that  the  deformable 
objects  can  be  handled  in  an  optimal  way  in  terms  of  certain  criteria?  This  problem  will  be 
studied  in  this  paper. 

In  reality,  there  are  three  fundamental  types  of  deformable  objects,  which  are  respectively, 
one,  two,  or  three  dimension'd.  The  first  two,  however,  occur  more  often  than  the  third. 
Furthermore,  two  dimensional  objects  ca"  often  be  treated  as  one  dimensional  when  it  is  picked 
up  by  two  manipulators.  Consider  a  rectangular  sheet  picked  up  by  two  manipulators  as 
shown  in  Fig.  1.  If  the  sheet  is  picked  up  in  such  a  way  that  the  end-effectors  are  positioned  at 
the  center  of  the  shorter  dimension,  the  deformation  is  more  likely  to  occur  in  the  direction  of 
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the  longer  dimension.  Once  the  deformation  occurs  in  one  dimension,  the  deformation  in  the 
other  dimension  becomes  negligible.  As  a  result,  two  dimensional  objects  can  be  treated  as  one 
dimensional.  For  the  reasons  just  cited,  we  concentrate  our  study  on  one  dimensional  objects, 
and  the  objects  are  simply  called  beams  in  the  rest  of  the  paper.  We  also  assume  that  the 
deformation  is  elastic.  That  is,  when  the  deformation  force  does  not  exist,  the  object  is  not 
deformed. 

In  the  next  section,  we  will  define  kinematic  constraints  for  two  coordinating  manipulators 
when  they  handle  an  elastic  beam.  Since  the  beam  is  deformable,  the  relative  position  and 
orientation  of  the  two  manipulators  are  not  necessary  constant  Instead,  the  constraint  relations 
can  be  described  by  inequality  equations.  This  gives  the  two  coordinated  manipulators  certain 
degrees  of  freedom  in  programming  their  motion  trajectories.  Their  relative  position  and 
orientation  can  be  modified  to  a  certain  extent  to  optimize  some  performance  criteria.  For 
example,  when  a  flexible  beam  is  folded  and  moved  by  two  manipulators,  the  required 
workspace  is  smaller  than  when  the  beam  is  fully  extended. 

In  the  third  section,  we  propose  three  methods  for  the  two  manipulators  to  handle  an 
elastic  beam.  In  the  first  method,  the  relative  orientation  can  be  altered,  but  not  the  position.  In 
the  second  method,  the  relative  position  of  the  two  end-effectors  can  be  altered,  but  not  the 
orientation.  In  the  third  method,  both  the  relative  position  and  orientation  can  be  altered. 
Through  the  study  of  the  object  mechanics,  the  first  method  is  quickly  eliminated.  Thus,  the 
study  concentrates  on  the  second  method.  This  method  is  investigated  in  terms  of  the 
magnitude  of  the  forces  and  moments  exerted  on  the  objects  and  end-effectors.  It  is  clear  that 
in  order  to  avoid  any  possible  damage  to  the  objects  and  heavy  loads  to  the  manipulators,  the 
reaction  forces  and  moments  between  the  objects  and  end-effectors  should  be  as  small  as 
possible. 

In  the  fourth  section,  the  study  is  concentrated  on  the  third  method.  It  is  determined  that 
the  third  method  results  in  less  reaction  forces  and  moments  than  the  second  method. 
However,  the  motion  trajectories  of  die  end-effectors  are  more  complex.  In  order  to  make  the 
motion  planning  feasible  in  practical  applications,  we  further  propose  an  approximation 
approach.  The  behavior  of  the  approximation  approach  is  also  analyzed. 

In  the  fifth  section,  experimental  results  using  both  the  second  and  third  methods  are 
presented.  The  paper  is  concluded  by  the  sixth  section,  Conclusions. 


11 


2.  KINEMATIC  CONSTRAINTS  ON  TWO  MANIPULATORS  HANDLING 

A  DEFORMABLE  BEAM 

In  this  section,  we  investigate  kinematic  constraints  imposed  on  the  relative  position  and 
orientation  of  two  manipulators  when  they  handle  a  flexible  beam,  for  which  the  following 
review  of  notations  of  the  manipulator  kinematics  is  necessary. 

According  to  the  Denavit-Hartenberg  convention  [22],  each  link  of  a  manipulator  is 
assigned  a  coordinate  system  (x-.y^z.)  for  i=0,l,...,n,  from  the  base  link  to  the  end-effector. 

The  generalized  coordinate,  is  the  joint  displacement  of  link  i  either  rotating  about  or  sliding 

along  Zj  j.  Let  a]_i  be  the  4  by  4  homogeneous  transformation  matrix  which  transforms  a 
vector  with  reference  to  coordinates  (Xj.y-.Zj)  to  the  same  vector  with  reference  to  coordinates 

(xi-i,  yi-i,  zj.i).  Then  one  may  have 

AJ  (q)^  (q,)A?  (q2)~A°  ,  (q„)  (1) 

n 

where  q  is  an  n-dimensional  vector  consisting  of  n  joint  displacements  qj,q2,. Aq  can  be 
represented  as  follows 

An  ,  ,  fn(q)  s(q)  a(q)  p(q)  1 

Ao  (q)=Lo  0  0  1  J  (2) 

where  n(q),  s(q)  and  a(q)  are  unit  vectors  of  coordinates  xn,yn  and  z^  respectively,  and  p(q)  is 
position  vector  (Fig.  2).  All  vectors  are  with  reference  to  the  base  coordinate.  The  orientation 
of  the  end-effector  is  specified  by  the  3X3  submatrix  of  Aq  (q): 

rJ5  (q)=[n(q)  s(q)  a(q)J  (3) 

Once  the  position  and  orientation  vectors  of  the  manipulator  end-effectors  are  determined,  the 
translational  and  angular  velocities  of  the  manipulator  end-effectors  v  and  <*  ran  be  derived 
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from  p(q)  and  Rq  (q)  [22].  Then  the  robot  joint  velocities  can  be  calculated  from  the  following 
equation 

q=J  '(q)  [^,  ]  (4) 

where  J(q)  is  the  Jacobian  matrix. 

We  are  now  in  a  position  to  determine  kinematic  constraints  imposed  on  the  two 
manipulators  which  handle  a  flexible  beam  (Fig.  2).  Consider  two  manipulators  each  with  n 
joints.  For  convenience,  one  of  the  manipulators  is  named  the  leader  and  the  other  the 
follower.  It  is  assumed  that  there  are  no  relative  motions  among  the  end-effectors  and  the 
beam,  and  the  length  of  the  beam  is  inextensible. 

Let  (xj,  ,  yj,  ,  zj,  )  and  (xj,  ,  yj,  ,  zj,  )  be  the  coordinate  systems  of  end-effectors  of  the 

leader  and  the  follower,  respectively.  Let  r*  be  the  vector  with  reference  to  (xj,  ,  yj,  ,  zj,  ) 

(Fig.  2).  Then  the  holonomic  constraints  on  the  relative  position  between  the  two  robots  can 
be  written  as  in  [4,  5] 

P(q')+Ro  (qV-p(qf)  =  0.  (5) 

From  (5),  one  may  have 
p(qf)  -  p(q')  =  Ro  (q1)  r1 

Since  the  beam  is  flexible,  the  distance  between  the  two  end-effectors,  i.e.,  r*  is  not  a  constant. 
As  a  result,  one  may  obtain  the  following  inequality  equation  from  (6): 

rmin<,P^f)-P(£l1>,=  |  *0  | =,rl,<  rmax  (7) 

where  Ir1!  represents  the  length  of  vector  r1 ,  and  rmin  and  rfnax  are  the  minimim  and  maximum 
length  of  Ir5!.  It  is  clear  rfflax  must  be  less  than  or  equal  to  the  total  length  of  the  beam. 
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Otherwise,  either  the  beam  is  tom  apart  or  the  contacts  between  the  beam  and  the  end-effectors 
are  lost,  both  of  which  are  clearly  not  desired.  To  determine  the  minimum  distance  rm^  is 

rather  involved,  which  will  be  studied  in  section  3. 

Eq.(7)  reveals  that  when  two  manipulators  are  coordinated  to  handle  a  deformable  object, 
either  one  or  two  dimensional,  the  manipulators  have  more  motion  freedom  than  when 
handling  a  rigid  object.  This  motion  freedom  can  be  utilized  to  optimize  other  performance 
criterion  of  the  manipulators.  For  example,  avoid  obstacles  while  handling  the  object,  or 
reduce  the  required  workspace  when  the  object  is  large  (to  be  realized  by  the  reduction  of  the 
distance  between  the  two  end-effectors),  etc.  The  detailed  utilization  of  this  advantage, 
however,  will  not  be  further  addressed  here. 

In  addition  to  the  flexibility  of  the  relative  position,  the  relative  orientation  is  also  allowed 
to  be  modified  in  certain  degree.  The  problem  is  how  to  formulate  this  motion  freedom. 
Recall  that  when  the  object  is  rigid,  the  holonomic  constraints  for  the  orientation  are  [4, 5] 

[RS  (q1)  ]T  Ro  (qf)  =  U  (8) 

where  U  is  the  constant  matrix.  When  the  two  manipulators  handle  a  flexible  beam  as  shown 
in  Fig.  2,  we  assume  that  twisting  of  the  beam  is  not  allowed.  That  is,  n(q*)  and  n(qf)  are 
always  parallel.  The  holonomic  constraints  imposed  on  n(q*)  and  n(q*)  can  be  described  by  the 
following  equation 

nT(qbn(qf)=l.  (9) 

However,  the  relative  orientation  between  the  two  vectors  a(q!)  and  a(qf)  can  be  modified, 
since  the  beam  can  be  bent  to  a  certain  extent  without  being  damaged.  The  holonomic 
constraints  imposed  on  a(ql)  and  a(qf)  can  thus  be  depicted  by  the  following  inequality 
equation: 


&min  <  aT(q')  a(q^)  <  a  max- 


(10) 
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It  is  clear  that  amin  is  equal  to  -1  which  occurs  when  the  beam  has  no  deformation,  and  amax  is 
less  than  1.  When  amax  is  equal  to  one,  the  relative  orientation  of  the  two  end-effectors  is 
allowed  to  be  zero.  This  will  bend  the  beam  as  much  as  90°  by  each  end-effector. 

In  this  section,  we  have  defined  kinematic  constraints  for  two  manipulators  handling  a 
deformable  beam.  It  has  been  pointed  out  that  the  relative  position  and  the  orientation  of  the 
two  end-effectors  can  be  modified  to  a  certain  extent  when  handling  deformable  beams.  How 
the  modification  can  be  made  and  what  the  relationship  between  the  modifications  of  the 
positions  and  orientations  remain  unanswered.  These  topics  will  be  studied  in  the  following 
sections. 


3.  HANDLING  A  DEFORMABLE  BEAM  BY  TWO  MANIPULATORS  WITH 

ONLY  POSITION  ALTERNATABLE 

In  this  section,  we  investigate  the  strategies  for  handling  a  deformable  beam  by  two 
manipulators.  The  strategies  will  define  the  limitations  of  rmin  and  amax  which  appear  in  (7) 
and  (10),  as  well  as  the  relationship  between  the  positional  and  orientational  modifications. 

From  the  discussion  of  the  previous  section,  it  can  be  easily  found  that  three  methods  are 
theoretically  possible  to  handle  a  flexible  beam.  They  are: 

Method  A:  the  relative  orientation  is  allowed  to  change,  but  the  relative  position  is  not. 

Method  B:  the  relative  position  is  allowed  to  change,  but  the  relative  orientation  is  not. 

Method  C:  both  the  relative  position  and  the  relative  orientation  are  allowed  to  change. 

With  three  methods  proposed,  it  is  natural  to  make  comparison  among  them  such  that  the 
best  one  can  be  determined.  Before  doing  so,  a  careful  look  at  Fig.  2  reveals  that  method  A 
is  impossible  in  reality.  Since  the  beam  is  firmly  grasped  by  the  end-effectors,  one  cannot 
change  the  relative  orientations  without  changing  the  relative  positions  unless  the  portions 
grasped  by  the  end-effectors  are  tom  off  from  the  beam.  The  latter,  however,  is  unrealistic. 
We  therefore  can  eliminate  method  A  and  concentrate  on  methods  B  and  C. 

To  evaluate  methods  B  and  C,  we  select  the  reaction  forces  and  moments  between  the 
beam  and  the  end-effectors  as  the  criterion.  This  selection  is  based  on  the  fact  that  the  forces 
and  moments  are  exerted  on  the  beam  when  the  beam  is  deformed,  and  should  be  as  small  as 
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possible.  Small  reaction  forces  and  moments  result  in  low  external  loads  to  the  manipulators  as 
well  as  low  possibilities  of  damage  to  the  objects.  This  consideration  amounts  to  analyzing  the 
deflection  behavior  of  the  beam  under  the  external  forces  and  moments  for  each  method.  In  the 
rest  of  this  section,  we  will  study  the  method  B.  Method  C  will  be  investigated  in  the  fourth 
section. 


3. 1  Deflection  Behavior  of  the  Beam  when  the  Beam  Has  a  Small  Deflection 

In  this  subsection,  we  analyze  the  deflection  behavior  of  the  beam  when  the  beam  has  a 
small  deflection.  It  is  first  assumed  that  the  beam  is  weightless.  This  assumption  is  based  on 
the  consideration  that  we  are  dealing  with  deformable  beams  whose  deformation  caused  by  the 
beam  weight  is  relatively  small.  Only  when  external  forces  and  moments  are  exerted  on  the 
beam  ends,  will  deformation  become  large.  In  reality,  many  deformable  objects  have  this 
characteristic,  a  practical  example  of  which  will  be  shown  in  the  experimental  studies. 

Consider  a  beam  held  by  the  end-effectors  at  its  two  ends.  For  convenience,  we  call  the 
line  that  connects  the  two  end  points  of  the  beam  the  reference  line.  When  the  beam  completely 
lies  on  the  reference  line,  no  deflection  occurs.  If  the  two  end-effectors  move  towards  each 
other  on  the  reference  line  with  the  orientation  unchanged,  the  beam  starts  to  be  deformed. 
According  to  classic  mechanics  [23,  24],  the  beam  can  be  deformed  in  two  different  cases. 
Fig.  3  shows  the  first  case  in  which  the  deformation  has  only  one  buckle,  and  Fig.  4  shows  the 
second  case  in  which  the  deformation  has  two  buckles  [23, 24]. 

In  case  1,  the  beam  is  deflected  to  form  a  curve  in  the  center,  but  the  beam  ends  are  still 
aligned  in  the  same  directions  of  the  end-effectors.  In  doing  so,  two  infection  points  appear  on 
the  beam.  The  curvatures  at  the  infection  points  are  zero;  therefore,  no  moments  are  exerted  on 
the  infection  points.  Thus  the  infection  points  are  also  called  the  points  of  zero  moment  (ZM) 
[23, 24].  It  is  also  pointed  out  in  [24]  that  the  points  of  ZM  occur  at  approximately  the  quarter 
points  of  the  beam  (Fig.  3). 

Now  just  consider  the  portion  of  the  beam  between  the  two  points  of  zero  moment.  Its 
length  is  one  half  of  the  entire  beam.  Since  the  points  of  ZM  do  not  provide  any  moment  to  the 
beam,  the  points  behave  as  hinges  (recall  that  the  hinges  do  not  provide  moments  to  the  beam 
but  only  the  forces).  Thus,  the  portion  between  the  two  points  of  ZM  behaves  as  a  hinged- 
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hinged  beam.  For  a  hinged-hinged  beam,  however,  the  relationship  between  the  external  force 
and  the  deflection  is  well  defined  (Appendix  1  of  this  paper  provides  the  details).  It  can  be 
shown  that  if  the  deflection  of  the  beam  is  small,  the  force  exerted  on  the  point  of  ZM  can  be 
expressed  as 


Fis  = 


7t2EIz  47C2EIz 
(L/2)2  =  L2 


(11) 


where  E  is  the  stiffness  of  the  beam  material,  Iz  the  moment  of  inertia  of  the  cross-section  of 

the  beam  with  respect  to  axis  z,  and  L  the  length  of  the  beam.  Since  the  deformation  is  static 
once  the  distance  between  the  two  end-effectors  becomes  constant,  the  end-effector  has  to 
provide  the  same  force,  i.e.,  Fis  to  the  beam  end.  In  addition,  if  the  deflection  of  the  hinged 
portion  is  5,  the  total  deflection  of  the  beam  will  be  28  [23, 24]  (Fig.  3).  That  is,  the  distance 
from  the  point  of  ZM  to  the  reference  line  is  8.  As  a  result,  the  moment  that  is  exerted  on  the 
end-effectors  can  be  expressed  as 

47E2EI  8 

MiS=f1s8  =  — (12) 


Note  that  the  deflection  parameter  8  is  very  small  in  a  small  deflected  beam,  and  no 

formulation  is  provided  to  calculate  it  In  the  large  deflection  case,  8  is  no  longer  negligible, 

and  needs  to  be  specifically  calculated.  This  topic  will  be  further  addressed  later. 

In  case  2,  it  is  found[24]  that  the  points  of  ZM  (i.e.  the  hinges)  occur  at  approximately  j 

0.7L  _  _  0.7L 

and  ~2~  points,  as  shown  in  Fig.  4.  Hence  the  portion  of  length  “y  is  in  effect  a  hinged- 

hinged  beam.  The  behavior  of  the  hinged-hinged  beam  in  this  case  is  also  provided  in 
Appendix  1,  and  we  find  the  force  exerted  on  the  hinge  to  be: 


T^EL  87t2EI 

_ z _ _ z 

(0.7L/2)2  L2 


(13) 
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From  (13)  and  Fig.  4,  one  can  also  obtain  that  the  force  exerted  on  the  end-effector  is  F2S.  and 
the  moment  exerted  on  the  end-effector  is: 

8rt2EI  8 

M2S  =  - j2-  (14) 

L 

Since  8  is  not  specified  in  (12)  and  (14),  the  absolute  value  of  Mis  and  M2S  can  not  be  calculated 
unless  one  knows  the  exact  deflection  in  reality.  However,  equations  (1 1)  -  (14)  reveal  that  for  the 
same  amount  of  deflection,  case  1  results  in  less  forces  and  moments  exerted  on  the  end-effectors. 
Therefore,  case  2  should  be  avoided  in  practice. 

In  reality,  if  the  beam  is  originally  grasped  by  the  end  effectors  with  no  deflection,  and 
the  end-effectors  start  to  approach  to  each  other  on  the  reference  line,  case  1  will  always  occur 
first.  This  is  because  the  deformation  forces  and  moments  gradually  grow  as  the  distance 
between  the  end-effectors  decreases,  and  case  1  needs  less  forces  and  moments  than  in  case  2. 
Once  case  1  occurs,  case  2  will  never  occur.  On  the  other  hand,  if  the  two  end-effectors  are  off 
the  reference  line,  i.e.,  the  two  vectors,  a(q),  of  the  end-effectors  are  not  aligned  on  the  same 
line,  case  2  may  occur  first  (Fig.  5).  Therefore,  to  avoid  case  2,  one  should  maintain  the  two 
end-effectors  on  the  same  reference  line  when  they  are  approaching  to  each  other. 

3.2  Deflection  Behavior  of  the  Beam  when  the  Beam  Has  a  Large  Deflection 

The  results  obtained  in  the  previous  subsection  are  for  small  deflections.  In  the  small 
deflection  case,  the  distance  between  the  two  end-effectors  are  virtually  the  same  as  the  length 
of  the  beam.  Therefore,  the  end-effectors  do  not  have  too  much  freedom  in  their  motion. 
When  the  end-effectors  are  even  closer  to  each  other,  the  beam  may  have  a  large  deflection. 
Although  the  forces  and  moments  exerted  on  the  end-effectors  will  be  different  in  the  large 
deflection  case,  the  analysis  of  the  forces  and  moments  are  still  rooted  in  the  small  deflection 
case. 

Introduce  a  fixed-free  ended  beam  under  large  deflection  as  shown  in  Fig.  6.  Note  that 
one  end  of  the  beam  is  fixed  on  the  wall  and  the  other  has  a  force  F3L  exerted  on  it.  It  is 

shown  in  Appendix  1  that  the  behavior  of  this  fixed-free  ended  beam  is  the  same  as  the  right 
half  of  a  hinged-hinged  beam  under  small  deflections.  The  convenience  of  this  introduction. 


t 
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however,  will  be  soon  made  clear.  It  is  shown  in  [24]  that  for  large  deflections,  the  force 
exerted  on  the  hinge  can  be  obtained  from  the  following  equation: 


Fj  *  (  n  )  (15) 

where  is  the  force  exerted  on  the  end  of  a  fixed-free  ended  beam  when  the  deformation  is 

small,  and  its  value  can  be  calculated  using  the  formulation  described  in  Appendix  1.  In  (15) 
R(Ot)  is  obtained  from  the  following  integral: 

x  f  d0 

R<a-<t,i)=  y—r.  a«> 

\  l-ksurd) 

0 

2  It 

where  k-sin  (Ct/2),  angle  a  is  the  deflection  angle  as  shown  in  Fig.  6.  When  ,  (16)  gives 

R(a).  Also,  for  large  deflections,  the  exact  deflection,  5,  can  be  calculated  by  the  following 
equation: 


R«X) 


sin((X/2) 


where  Lj  is  the  length  of  the  beam.  Furthermore,  the  distance,  rj ,  can  be  obtained  from 


rl  2P(q) 


Li  R(Ot) 


where  P(a)  can  be  calculated  from  another  elliptic  integral: 


•  I 

P(CC,<}>1)=  JV l-ksin2(|)  d<j). 
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From  (19),  one  may  have  P(CC)  =P  (CC,^)  | 


With  the  forthgoing  analysis  of  the  fixed-free  ended  beam,  we  can  now  further  evaluate 
Method  B.  From  Fig.  3,  it  can  be  seen  that  in  case  1,  the  entire  beam  can  be  divided  into  four 
pieces  of  fixed-free  ended  beam,  separated  by  the  points  of  ZM.  Starting  from  the  left,  the  first 
piece  is  from  the  left  end-effector  to  the  first  point  of  ZM  (consider  that  the  end-effector  as  the 
fixed  point),  and  the  second  piece  is  between  the  first  point  of  ZM  and  the  center  point  (the 
center  point  can  be  considered  as  the  fixed  point),  etc.  For  every  piece,  we  can  determine  the 
hinged  force  F3L,  the  deflection  5  and  the  distance  Tj,  from  the  deflection  angle  a  (here  a  is 

the  same  as  0  shown  in  Fig.  3)  using  (15),  (17)  and  (18).  It  should  be  noted  that  Lj  in  (17) 

o  .  L 

and  (18)  is  equal  to  4  .  It  is  clear  that  the  hinged  force  is  also  the  force  exerted  on  the  end- 

effector.  The  total  deflection  of  the  beam,  however,  is  twice  as  much  as  the  calculated 
deflection  5,  and  the  distance  between  the  two  end-effectors,  Ir1!,  is  four  times  as  much  as  Tj. 

Based  on  the  above  discussion,  the  behavior  of  the  large  deflection  beam,  as  shown  in 
Fig.  3,  can  be  expressed  as  follows  (here  F„ ,  the  force  under  large  deflection,  is  used  instead 

of  F1S) 


Fnr 


(4R(CC))ZEI 


Ir1!  =L(^^  -  1) 
R(OC) 


and  the  deflection  of  the  beam  is  equal  to 


28  =  ^  sin  (y  ) 

R(a)  2 


The  moment  exerted  on  the  end-effector  can  be  calculated  as 


MlL  =  FllA 
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One  may  note  that  the  evaluation  of  (20)  to  (22)  involves  the  computation  of  two  elliptic 
integrals.  But  in  reality,  R((X)  and  P(O0,  corresponding  to  a  certain  deflection  angle  (X,  can  be 
found  from  a  well  developed  table  which  is  provided  by  many  textbooks  of  mathematics  and 
mechanics.  For  example,  the  table  shown  in  Appendix  2  is  found  in  reference  [25]. 


3.3  The  Minimum  Distance  between  the  Two  Grippers 

With  the  results  provided  in  the  previous  two  subsections,  we  can  further  investigate  the 

minimum  distance  between  the  two  end-effectors.  The  criterion  to  determine  the  minimum 

distance  r  .  is  based  on  two  factors:  the  maximum  stress  that  the  deformable  beam  can 
min 

withstand  without  being  damaged,  and  the  maximum  load  that  the  manipulators  can  carry. 
The  procedure  of  the  determination  is  as  follows. 

Consider  a  single  piece  ended  by  a  ZM  point  and  die  end-effector  in  an  entire  beam 
(Fig.  3).  It  behaves  as  a  fixed-free  ended  beam,  for  which  the  following  relationship  must 
always  hold  [24] 


_  1L  . 

°direct'K>bending~  A 


F..  ^lL8 


<o. 


max 


(24) 


where  Ofnax  is  the  maximum  allowable  stress  that  the  beam  can  withstand,  is  the  stress 

due  to  direct  compression,  and  ^^nding  is  the  stress  due  to  bending.  A  is  the  area  of  the 
cross  section  of  the  beam,  Iz  is  the  area  moment  of  inertia  with  respect  to  axis  z,  and  ym  is  the 

distance  from  the  center  to  the  edge  of  the  area  where  the  bending  occurs  (Fig.7).  Since  both 
F^  and  8  are  functions  of  the  bending  angle  0t,  finding  the  minimum  distance  amounts  to 

finding  the  angle  a  such  that  (24)  is  satisfied.  However,  because  both  the  functions  involve 
elliptic  integrals,  solving  for  a  from  (24)  is  rather  complicated.  Instead,  we  can  use  a  trial- 
and-error  method. 

We  may  first  arbitrarily  select  a  deflection  angle  (X.  Based  on  (X,  FJL  can  be  calculated 
using  (20),  and  the  deflection  8  can  be  calculated  using  (22).  Substituting  the  calculated  F1L 
and  8  into  (24).  If  it  can  be  satisfied,  we  further  enlarge  a;  otherwise  it  should  be  reduced. 
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The  same  procedure  is  then  tried  again  until  a  maximum  deflection  angle  which  satisfies  (24)  is 
found. 

The  hinged  force  and  moments  thus  found  still  have  to  satisfy  the  load  criterion  of  the 
manipulator.  If  MlL  and  F1L  are  less  than  the  maximum  capabilities  of  the  manipulators,  the 

selected  deflection  angle  is  acceptable.  Otherwise,  they  have  to  be  reduced.  Basically,  if  the 
calculated  force  is  greater  than  the  capability  of  the  manipulators,  we  have  to  use  (20)  to  find 
the  maximum  value  of  R(CC)  Get  FJL  be  the  capability  of  the  manipulator),  and  further  find  the 

deflection  angle  from  the  lookup  table.  Likewise,  (23)  may  be  used  to  determine  the  deflection 
angle  if  the  calculated  MJLis  greater  than  the  capability  of  the  manipulator.  Once  the 

maximum  deflection  angle  is  found,  one  may  use  (21)  to  find  the  minimum  distance  that  the 
two  end-effectors  can  be  at  while  handling  a  flexible  beam. 

4.  HANDLING  A  DEFORMABLE  BEAM  BY  TWO  MANIPULATORS  WITH 
BOTH  ORIENTATION  AND  POSITION  ALTERNATABLE 

In  the  previous  section,  we  have  studied  method  B  in  which  the  end-effectors  are  only 
allowed  to  have  position  difference.  In  this  section,  we  investigate  method  C  in  which  both 
position  and  orientation  alterations  are  permitted.  It  is  clear  that  there  arc  many  possible 
combinations  of  the  relative  position  and  orientation  between  the  two  end-effectors.  However, 
not  all  of  them  are  suitable  for  handling  deformable  beams.  We  should  first  select  the  best 
combination  from  all  the  possibilities.  Then  the  minimum  distance  and  the  maximum 
orientation  difference  between  the  two  end-effectors  can  further  be  investigated. 

4.1  The  Best  Combination  of  the  Relative  Position  and  Orientation 

To  determine  the  best  combination,  we  again  use  the  force  and  moment  criterion,  i.e, 
minimizing  the  force  and  moment  exerted  on  the  end-effectors  as  well  as  on  the  objects.  To  do 
so,  we  propose  and  prove  the  following  claim. 

Claim:  When  the  two  end-effectors  grasp  the  beam  in  such  a  way  that  the  two  end  points  of 
the  beam  coincide  with  the  ZM  points  of  the  beam,  and  the  orientation  of  the  end-effectors  are 
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the  same  as  the  deflection  angle  of  the  beam  (Fig. 8),  the  end-effectors  suffer  the  minimum 
forces  and  moments. 

Proof:  We  prove  this  claim  for  the  forces  and  the  moments,  respectively,  as  follows.  From 
the  discussion  of  the  previous  section,  it  can  be  seen  that  the  force  exerted  on  the  ZM  point  is 
related  to  the  length  of  the  portion  between  the  two  ZM  points.  Longer  length  results  in  small 
forces  in  both  the  small  and  large  deflection  cases  (see  (11)  and  (20)).  It  follows  that  if  the  two 
end  points  of  the  beam  are  at  the  ZM  points  while  the  beam  is  bent,  the  end  points  will  behave 
as  hingers.  Since  the  length  of  the  beam  is  the  maximum  distance  that  the  portion  between  the 
two  hinge  points  could  be,  the  force  exerted  on  the  hinge  points  or  the  center  of  the  end- 
effectors  (assume  that  the  center  of  the  end-effectors  coincide  with  the  end  points  of  the  beam) 
becomes  the  minimum. 

Next  we  consider  the  moments  exerted  on  the  end-effectors.  Since  the  orientations  of  the 
end-effectors  coincide  with  the  deflection  angle  of  the  beam,  the  curvature  of  the  beam  at  the  tip 
of  the  end-effectors  are  null.  If  the  curvature  is  zero,  it  follows  from  the  following 
formulation,  which  describes  the  deformation  behavior  of  a  elastic  beam  [23,24], 


1 

P 


(25) 


that  the  moments  exerted  on  the  end-effectors  are  zero,  where  is  the  curvature  of  the  beam. 


Any  deviation  of  the  end-effectors  orientation  from  the  deflection  angle  will  result  in  non- zero 
curvature  which  will  generate  bending  moments  exerted  on  the  end-effectors.  ♦ 

With  the  claim  just  proved,  we  can  further  consider  the  minimum  distance  and  the 
maximum  orientation  difference  between  the  two  end-effectors.  This  will  be  discussed  in  the 
following  subsection. 


4.2  The  Minimum  Distance  and  the  Maximum  Orientation  Difference  between  the  Two  End- 
Effectors 


The  procedure  applied  in  subsections  3.2  and  3.3  can  be  used  again  here.  The  beam  now 
behaves  as  two  pieces  of  fixed-free  ended  beam  (Fig.  8).  The  parameters  associated  with  the 
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deformation  can  be  respectively  calculated  as  follows.  First,  the  exerted  force  can  be  written 

as: 


(2R(a))2EI 

F  = - 5 - 5  .  (26) 

4L  Lr 


Then,  the  distance  between  the  two  end-effectors  can  be  calculated  using: 


lr1! 


U2PM 

(R(a) 


l), 


(27) 


and  the  deflecion  of  the  beam  is  equal  to: 


5  = 


L 

2R«X) 


sin  (y ) . 


(28) 


The  minimum  distance  between  the  two  end-effectors  can  be  determined  using  the  procedure  as 
outlined  in  subsection  3.3  using  (26),  (27)  and  (28)  instead  of  (20),  (21)  and  (22). 
Essentially,  one  needs  to  find  a  maximum  deflection  angle,  Otniax-  Once  Oma*  is  found,  not 
only  the  minimum  distance  can  be  found  by  using  (27),  but  also  the  maximum  orientation 
difference  can  be  obtained  as: 

aT(q')a(qf)  ^  &max  =  cos(20tmax)*  (29) 

In  reality,  however,  the  exact  value  of  the  maximum  stress  is  often  not  known;  therefore, 
mathematical  computation  of  the  minimum  distance  and  the  maximum  orientation  difference 
may  not  be  possible.  In  this  case,  we  may  have  to  practically  bend  the  beam  until  the  limit  is 
reached.  In  this  regard,  specifying  the  trajectories  of  the  end-effectors  while  they  arc  bending 
the  beam  becomes  a  more  important  issue  than  finding  the  maximum  orientation  difference. 
This  topic  will  be  discussed  in  the  next  subsection. 
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4.3  The  Relationship  between  the  Positions  and  Orientations  of  the  End-Effectors  while 
Bending  the  Beam 

For  simplicity,  we  concentrate  our  discussion  on  a  single  end-effector  which  is  shown  on 
the  right  side  of  Fig.  8.  Assume  that  when  the  end-effector  picks  up  the  beam,  the  beam  is  not 
deformed,  and  the  position  and  orientation  of  the  end-effector  is  [x,  y,  CX]  =[0, 0, 0].  From  the 
discussion  of  the  previous  subsection,  it  can  be  found  that  the  trajectory  of  the  end-effector  can 
be  expressed  as: 


f  P«X)  l 

,  x(a)=L(R(Ct)  '  2) 

ly(a)i^sin((X/2) 


(30) 


Note  that  the  x  and  y  coordinates  are  complex  functions  of  the  end-effector  orientation. 
Essentially,  we  can  use  the  end-effector  orientation  as  an  independent  variable,  and  linearly 
increase  it  as  the  end-effector  bends  the  beam.  The  position  of  the  end-effector  can  be 
calculated  by  using  (30). 

There  are  two  practical  problems  involved  in  this  method.  First,  the  computation  involved 
in  (30)  is  rather  complicated.  It  is  practically  impossible  to  execute  the  computation  of  (30)  in 
real  time  by  the  manipulator  controller.  Secondly,  even  if  the  computation  can  be  handled  by 
the  controller,  for  example  by  using  a  look-up  table  as  shown  in  Appendix  2  (a  much  more 
detailed  version  is  needed  though),  executing  such  a  trajectory  by  the  practical  manipulators  is 
very  time  consuming,  since  the  manipulators  have  to  execute  point  to  point  motion  along  the 
trajectory.  Between  every  pair  of  points,  the  motions  have  to  be  interpolated  by  a  complicated 
compulation  [25].  If  one  can  reduce  the  intermediate  points  on  the  trajectory,  the  computation 
can  be  simplified,  and  the  manipulator  motion  can  be  sped  up. 

In  reality,  we  coarsely  divide  the  desired  trajectories  of  the  two  end-effectors  into  a  few 
pieces.  For  example,  every  piece  can  cover  about  10  degrees  of  the  deflection  angle.  The 
positions  of  the  two  end  points  of  each  piece  can  be  specified  by  (30)  or  using  a  look-up  table. 
The  motions  between  the  two  end  points  are  simply  linear.  When  using  a  PUMA  manipulator. 
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this  motion  can  be  programmed  by  using  the  instruction  MOVES  D(n),  where  D(n)  is  the  end 
point  of  the  nth  piece.  The  instruction  means  that  the  end-effector  is  linearly  moved  from  the 
end  point  D(n-l)  to  D(n).  Since  the  trajectory  between  any  two  end  points  are  not  obtained 
from  (30)  or  a  look-up  table,  the  problem  is  how  the  force  and  moment  situation  will  be 
affected  by  using  this  simple  approximation  method.  This  topic  will  be  studied  in  the 
following  subsection. 

4.4  Behavior  of  the  Approximation  Method 

For  simplicity,  we  just  consider  one  half  of  the  entire  beam.  Assume  that  the  end-effector 
moves  along  the  trajectory  specified  by  (30),  and  the  grasping  point  is  located  at  point  Q  on  the 
trajectory  which  is  a  curve  between  the  two  end-points  N  and  M  (see  Fig.  9).  If  we  use  the 
line  segment  between  N  and  M,  instead  of  the  curve,  as  the  trajectory,  the  grasping  point  will 
be  at  J  as  shown  in  Fig.  9.  Without  loss  of  generality,  we  assume  that  JQ  is  parallel  to  the  axis 
x.  When  the  grasping  point  moves  from  point  Q  to  J,  the  shape  of  the  beam  is  converted  from 
OQ  to  be  O'J  as  shown  in  Fig.  10,  and  the  inflection  points  will  occur  on  the  beam  instead  at 
the  end  points  of  the  beam.  The  question  is  where  those  infection  points  are  since  the  positions 
of  the  infection  points  affect  the  forces  and  moments  exerted  on  the  end-effetors. 

As  mentioned  in  the  previous  section,  the  inflection  point  is  the  point  of  ZM,  and  the  ZM 
points  occur  at  the  quarter  points  of  the  beam  when  the  end-effectors  are  aligned  on  the  same 
line.  To  take  advantage  of  this  conclusion,  we  extend  the  length  of  the  original  beam  in  such  a 
way  that  the  infection  points  of  the  original  beam  are  also  at  the  quarter  points  of  the  extended 
beam.  Clearly,  for  the  extended  beam,  the  orientation  of  its  two  ends  must  be  aligned  on  the 
same  line.  For  the  right  half  of  the  extended  beam,  the  deformed  shape  is  shown  as  curve  OT1 
in  Fig.  10.  It  should  be  noted  that  the  original  beam  O'J  is  now  a  segment  of  the  extended 
beam  OH.  For  the  extended  beam  OH,  the  results  of  the  previous  section  can  be  used  in  our 
analysis. 

Assume  that  the  angle  between  the  end-effector  and  JQ  is  <p,  and  the  deflection  angle  at 
point  G  ( i.e.,  ZM  point )  is  (Xx  .  For  segment  GH  of  the  extended  beam  OH,  we  may  have 

the  following  relationships  [24]: 
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(31) 


sm  (<p/Z) 
Sin  *1-  sin  (ax/2) 


(32) 

(33) 

where  Lx  is  one  half  of  the  length  of  the  extended  beam  OH ;  t  and  s  arc  as  shown  in  Fig.  10 

(also  see  Fig.  6),  and  the  other  terms  have  previously  been  defined.  For  the  extended  beam, 
(18)  can  now  be  written  as: 


j  2P(cg 

V  R<cg 


(34) 


where  rx  is  as  shown  in  Fig.  10. 

From  (32)  and  (34),  we  obtain  the  following  expression  for  t : 

2P(ax,<{>1)-R(ax,4)1) 

1  =  r* (  2P(ax)  -  R(ax)  ) 

Let 

c  =  rx  +  (rx-t)  =  2rx-t. 

From  Fig.  10  we  also  have 

|  =  2Lx-s  (37) 

where  L  is  the  length  of  the  original  beam.  Substituting  (33)  into  (37),  we  obtain: 


(35) 

(36) 
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(38) 


LR(ax) 

Lx  =  4R(ax)-2R(ax,<))1) 


From  (34),  (35),  (36),  and  (38),  we  obtain: 

r_2f,(°x)-P(M,)  I  (39) 

L~2R(a:t)-R(a]1.<|>1)'2 ' 

From  the  discussion  of  subsection  4.2,  it  is  known  that  if  the  grasping  point  is  at  Q,  one 
has  (see  Fig.8  and  (18)): 

Il_  P(gO  .]  (40) 

L  R(nt) 

Let  the  length  of  JQ  be  A.  C  that  is  expressed  by  (36)  can  also  be  expressed  as: 

C  =  r,-A  (41) 

Using  (40)  and  (41),  (39)  becomes: 

A  P(cx)  2p((V-P(Mi>  (42) 

L  R(oc)  2R(ax)-R(ax,<}>1) 

From  the  above  discussion,  one  can  see  that  if  A  and  cp  are  specified,  CXx  can  be  calculated 

using  (31)  and  (42).  A  is  the  positron  difference  of  the  end-effector  between  points  Q  and  J, 
and  cp  can  be  assumed  to  be  the  same  as  (X  (recall  that  (X  is  an  independent  variable  when 
programming  the  motion  trajectory).  Once  CXx  is  calculated,  one  can  proceed  to  find  the  forces 

and  moments  exerted  on  the  end-effectors. 

For  the  extended  beam  OH  (Fig.  10),  one  can  use  (15)  to  determine  the  force  F^  that  is 
exerted  on  the  point  of  ZM,  G,  to  be: 

...  (Rta^Ei 
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It  is  clear  that  the  force  F  is  also  the  force  exerted  on  the  end-effector. 

x 

When  the  grasping  point  is  at  the  point  Q,  the  exerted  force  F4L  can  be  calculated  from  (26). 
From  (26),  (38;,  and  (43)  we  obtain  tne  following  relationship  between  Fx  and  F4L: 


2R(a  )  -  R((X  $,) 

_  / - 5 - 5 — -y- 

K  R(a)  '  ' 


(44) 


Since  R((Xx)  is  greater  than  R(ax,<j)j)  (the  former  is  equal  to  the  latter  only  when  <J>j  is  equal  to 
7C/2),  and  R((X  )  is  greater  than  R(Ot),  F  is  greater  thanF.. .  However,  as  long  as  A  is 
small,  the  difference  between  F^  and  F4L  is  still  manageable. 

To  calculate  the  moment  exerted  on  the  end-effector,  we  have  to  calculate  the  force  arm,  h, 
as  shown  in  Fig.  10.  From  [24],  it  is  known  that  the  relationship  between  h  and  Lx  as  shown 
in  Fig.  10  can  be  expressed  as: 

2L^  (cosep  -  cosax )  =  ((R(ax))2  h2  (45) 


Substituting  Lx  in  (45)  by  (38),  the  force  arm  can  be  written  as: 

h  = 

Consequently,  the  moment  exerted  on  the  end-effector  can  be  written  as  F^  h  and  calculated  by 

using  (43)  and  (46).  It  should  be  emphasized  that  the  moment  is  a  completely  new  addition  to 
the  total  forces  and  moments  that  are  exerted  on  the  end-effector. 

To  summarize  the  discussion  presented  so  far  in  this  section,  it  can  be  seen  that  because 
of  the  position  difference  caused  by  using  the  line  segment  to  approximate  the  trajectory  curve, 
the  forces  and  the  moments  exerted  on  the  end-effectors  are  enlarged.  If  it  is  necessary,  the 
exact  values  of  the  forces  and  moments  can  be  calculated  by  using  the  formulations  that  have 
just  been  defined.  In  the  next  subsection,  we  will  use  a  numerical  example  to  illustrate  how  the 
forces  and  moments  are  affected  by  the  position  difference  of  the  end-effectors. 


L'y  2(cos<p-cosax ) 
2(2R(ax)-R(ax,<j>1)) 
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4.5  Numerical  Example  for  the  Approximation  Method 


Consider  that  the  arc,  MN,  in  Fig.  9  is  the  trajectory  of  an  end-effector  specified  by  (30). 
With  the  specified  trajectory,  the  end-effector  bends  the  beam  from  OC  =  0°  to  GC  =10°. 

The  equation  for  the  line  segment  MN  can  be  expressed  by  is: 


L 

x  ~  2 

x(10°)  -  \ 


y(io°) 


where  x(10°)  and  y(10°)  can  be  calculated  by  using  (30). 

We  now  assume  that  at  point  Q,  a  =  5°,  which  is  in  the  middle  of  the  bending  process. 
The  reason  for  selecting  5°  is  that  at  the  middle  point,  the  difference  between  point  Q  and 
point  J  is  close  to  be  the  maximum,  and  the  forces  and  moments  are  close  to  the  maximum  as 
well. 

The  coordinates  of  point  Q  can  be  calculated  from  (30).  That  is: 


j  > 

y(5°)=R^jSin(5°/2) 


The  coordinates  of  point  J,  on  the  other  hand,  can  be  calculated  as: 


x  =  j£5!L(x(io0)- j  ) 

J  1  y(10  )  L 

yj  -  y(5°) 


Once  x(5°)  and  Xj  are  calculated,  the  difference,  A,  can  be  expressed  as: 
A  =  x(5°)  -  Xj 


The  numerical  values  are  calculated  as  follows: 
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8.9193  X  ltr4,  ax  «  5.5£ 


F 

pT-»  1.63,  and 
F4L 


It  can  be  seen  that  the  force  exerted  on  the  end-effector,  F  ,  is  1.63  times  as  much  as  F._  due 

x  4L 

to  the  difference  A.  If  the  trajectory  represented  by  the  curve,  MN,  is  approximated  by  more 
but  shorter  line  segments,  the  force  and  the  moment  can  be  reduced. 


5.  EXPERIMENTAL  STUDY 


To  verify  the  theoretical  results,  we  have  conducted  experiments  in  our  laboratory.  The  setup 
of  the  experiement  is  as  shown  in  Fig.  11.  Basically,  in  the  experiment,  a  large  number  of  elastic 
sheets  needed  to  be  assembled  together  to  form  a  book  The  book  is  then  to  be  laminated  to  form  a 
board.  This  assembly  procedure  is  typical  in  the  manufacturing  of  a  Printed  Wiring  Board  (PWB). 
Since  each  sheet  has  delicate  circuits  printed  on  the  surface,  and  the  circuits  cm  multiple  surfaces 
must  be  precisely  aligned  (within  0.001  inch),  the  assembly  of  the  book  is  not  a  simple  issue. 

For  the  alignment  purpose,  each  sheet  has  four  positioning  holes.  Those  holes  are  used  to 
align  the  sheet  with  pins  which  are  fixed  on  a  base-board,  which  is  used  to  hold  the  book  (Fig.  11). 
Since  the  tolerance  between  the  pin  and  the  hole  is  very  tight,  it  is  difficult  to  align  all  the  holes 
with  the  pins  at  the  same  time.  Instead,  in  the  assembly  process,  the  manipulators  bend  the  sheet, 
such  that  the  two  middle  holes  can  make  contact  with  the  pins  first.  Once  the  middle  holes  are 
aligned  with  the  middle  pins,  the  manipulators  extend  the  sheet  such  that  the  remaining  holes  are 
aligned  with  the  pins.  In  this  process,  the  two  manipulators  first  bend  the  sheet,  following  the 
trajectories  that  have  to  be  specified  using  the  methods  defined  in  the  previous  sections,  and  come 
back  to  their  original  positions  using  the  same  trajectories. 

We  have  tested  both  method  B  and  method  C  in  the  experiment.  In  each  method,  the  required 
bending  angle  was  40°.  From  the  table  shown  in  Appendix  2,  we  can  see  that  the  distance 
between  the  two  end-effectors  needs  to  be  reduced  by  about  12%.  Since  the  dimension  of  the 
sheet  was  18  by  12  inches,  and  the  end-effectors  held  the  center  of  the  short  sides,  this  reduction 
was  equivalent  to  2.1  inches.  The  experiment  was  conducted  using  an  aluminum  sheet  (it  was 
used  to  cover  and  protect  the  PWB  during  the  lamination  process).  When  method  B  was  used,  we 
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found  that  some  permanent  damage  was  made  near  the  tip  of  the  end-effectors.  This  is  because 
method  B  imposes  a  large  force  and  bending  moment  on  the  end-effector  as  discussed  earlier.  This 
clearly  shows  that  method  B  was  not  adequate  to  bend  the  elastic  sheet. 

We  then  turned  to  method  C.  In  programming  the  motions  of  the  end-effectors,  we  divided 
the  40°  deflection  into  four  small  pieces  with  10°  in  each  piece.  Using  the  look-up  table  as  shown 
in  Appendix  2,  we  specified  the  positions  of  the  end-effectors  at  the  end  points  of  the  pieces,  and 
used  those  points  as  the  destination  points  for  the  end-effectors  to  move.  Using  the  MOVES 
instructions  as  mentioned  earlier,  the  end-effectors  executed  a  piece-wise  linear  motion  to  move  the 
two  end-effectors  to  their  final  destination  points  at  which  the  sheet  was  bent  by  40°.  By  using 
method  C  the  assembly  was  successfully  completed  without  any  damage  to  the  aluminum  sheet 
(Fig.  12). 


6.  CONCLUSIONS 

In  this  paper  we  have  discussed  the  mechanism  for  two  coordinated  manipulators  to 
automatically  handle  deformable  objects.  After  the  kinematic  constraints  were  analyzed  for  the 
manipulators,  we  investigated  the  coordinating  methods  for  the  two  manipulators.  In  this  respect, 
the  main  contributions  can  be  summarized  as  follows: 

(1)  We  have  identified  two  coordinating  methods  for  the  two  manipulators  to  handle  one¬ 
dimensional  objects.  The  first  method  only  allows  the  relative  position  to  be  altered  between  the 
two  end-effectors.  The  second  method  allows  both  relative  position  and  orientation  to  be  altered. 
From  the  analysis,  we  identified  that  the  second  method  is  better  than  the  first  method. 

(2)  For  the  second  method,  we  further  specified  the  end-effector  trajectories,  while  they  are 
bending  an  elastic  sheet.  The  trajectories,  however,  were  very  complicated  to  compute.  We  further 
developed  an  approximation  method,  which  used  piece-wise  linear  segments  to  substitute  for  the 
trajectories.  Furthermore,  the  behavior  of  the  approximation  method  was  also  analyzed.  Finally, 
experimental  results  were  given  to  prove  that  the  second  method  as  well  as  the  approximation  of 
the  method  were  effective  for  handling  deformable  objects. 
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Appendix  1 


Consider  a  hinged-hinged  beam  (Fig.  A-l)  that  is  weightless  and  with  a  uniform  EIz,  and 
the  length  of  which,  L ^  is  inextensible.  According  to  Bemoulli-Euler  law  under  the  condition 

of  small  deflection,  we  may  have  [23,  24]: 
d^v  M 

dx2='EI*  <A-*> 


where  v  is  the  deflection  in  the  direction  y,  M  is  the  bending  moment  at  any  point  t  on  the 
beam,  and  EIz  is  the  beam  stiffness.  In  Fig.  A-l,  F  is  a  horizontal  force  exerted  on  the  end  of 

the  beam,  and  8=v 

max 

In  this  case,  one  has: 


M=Fv. 

(A-2) 

Thus  (A-l)  becomes 

d2v  Fv 
dx2='EIz  • 

(A-3) 

Let 

F  2 
ei,-k  ■ 

(A-4) 

Eq.  (A-3)  can  be  written  as 

d2v  ^ 

~2  +K2v=0. 
dx 

(A-5) 

Now  we  verify  that 
v=Cj  sin  Kx 

satisfies  equation  (A-5),  where  Cj  is  a  constant,  not  equal  to  zero. 

(A-6) 

It  is  easy  to  see  that  the  boundary  conditions  of  the  beam  are 
v=0  at  x=0 

(A-7) 

and 
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v=0  at  x=L2  . 

(A-8) 

The  second  boundary  condition  requires  that 

0=Cj  sin  KLj . 

(A-9) 

This  condition  will  be  satisfied  only  if 

KL2=n7C,  n=l,2, ... 

(A- 10) 

By  using  Eq.  (A-4),  we  can  get 

F_n2*H 

r=n  2 

L2 

(A- 11) 

Now  consider  two  subcases: 

n2Elz 

subcase  1  (Fig  A-2):  whenn=l,  one  has  Fj=  2  .  and 

L2 

(A- 12) 

47t2EI 

subcase  2  (Fig.  A-3):  when  n=2,  one  has  F,=  2 

Lz 

(A-13) 

We  can  employ  the  results  obtained  so  far  to  find  solutions  for  other  support  conditions  in 

the  following  manner.  Consider  a  fixed-free  ended  beam  as  shown  in  Fig.  A-4a.  It  has  a 
length  and  a  constant  EIZ.  In  Fig.  A-4b,  we  show  a  hinged-hinged  beam  of  length  2L^  and 

with  the  same  stiffness  constant,  EIz,  as  the  fixed-free  ended  beam.  It  can  be  seen  that  one-half 


of  the  hinged-hinged  beam  is  identical  to  the  fixed-free  beam.  Hence 
7C2EIZ 

Fr<V' 


(A- 14) 


i.e.. 


F3= 


7C2EIz 

4lJ 


(A-15) 
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Appendix  2 


LOOK-UP  TABLE  FOR  COMPUTING  THE  END-EFFECTOR 

POSITIONS 


<X(o) 

R«X) 

P«X) 

F3L 

F3 

6 

L1 

rl 

Ll 

0 

1.5708 

1.5708 

1.000 

0 

1 

10 

1.5738 

1.5678 

1.004 

0.1116 

0.9924 

20 

1.5828 

1.55.) 

1.016 

0.2193 

0.9698 

30 

1.5981 

1.5442 

1.035 

0.3239 

0.9325 

40 

1.6200 

1.5238 

1.062 

0.4221 

0.8812 

50 

1.6490 

1.4981 

1.102 

0.5126 

0.8170 

60 

1.6858 

1.4675 

1.152 

0.5930 

0.7410 

70 

1.7312 

1.4323 

1.215 

0.6626 

0.6547 

80 

1.7868 

1.3931 

1.294 

0.7195 

0.5593 

90 

1.8541 

1.3506 

1.392 

0.7625 

0.4569 

37 


Fig.  1  A  rectangular  sheet  picked  up 
by  two  manipulators 


Fig.  2  The  coordinate  system  and  notations 
for  two  manipulators 


Fig.  7  The  cross  section  of  a  beam 


Fig.  8  The  case  of  the  minimum  of  the  force  and  moment 
when  the  beam  is  bent 
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Fig.  9  The  curve  and  the  line  segments 
of  the  end-effector  trajectory 
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X  ‘X 


Fig.  1 0  Error  analysis  for  the  case  of  Fig.8 
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The  physical  experiments 


F 


v 


8 


F 


Fig.  A-3  The  beam  has  a  deformation 
with  two  buckles 


b) 


Fig.  A-4  A  fixed-free  ended  beam  under  small  deflection 


Appendix  B 


Determination  of  the  Third  Dimension  of  an  Object  by  Using 

a  Range  Finder 


Rached  Zantout  &  Prof.  Yuan  Zheng 


I  •  Eg, jut _ Location _ Using _ a  Range-Finder 


The  goal  is  to  locate  a  point  in  space  (i.e.  know  its  x,  y 
and  z  coordinates  with  respect  to  a  reference  frame)  knowing  only 
information  about  its  image,  captured  by  a  Range-Finder. 

The  Range-Finder  consists  of  a  camera  and  a  laser  beam 
generator . 

In  the  following,  the  system  set-up  and  calibration  will  be 
described.  Then  a  way  to  determine  the  coordinates  of  a  point  of 
interest  is  described.  Finally,  examples  are  given  to  prove  the 
validity  of  the  described  algorithm. 

S£t-.up _ and _ Calibration 


First,  the  camera  and  the  Laser-beam  generator  are  fixed  and 
a  reference  frame  is  chosen  such  that  its  origin  (0)  is  mapped  to 
the  middle  of  the  screen  (O' ) .  The  location  of  the  camera 
F(fx,fy,fz),  as  well  as  the  equation  of  the  scanning  plane 

(Ax+By+C*=D)  are  measured  with  respect  to  the  reference  frame. 
Then,  a  cube  is  put  in  front  of  the  camera  with  one  of  its  faces 
parallel  to  the  camera's  lens  plane.  A  point  (A)  on  that  face  is 
identified  and  the  distance  OA  measured.  On  the  screen,  the 
distance  between  the  images  of  those  two  points  (O'  and  A' )  is 
also  measured. 


f  =  OF  x 


OA 

OA' 


The  orientation  of  the  camera  is  then  measured,  and  the 
parameters  in  the  following  equations  are  determined. 

h  =  hx.x  +  hy.y  +  hz.x 

v  =  vx.x  +  Vy.y  +  vz.x 

»=ax.x  +  ay.y  +  az.« 
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h  and  v  define  the  coordinate  frame  of  the  image-plane, 
a  is  the  unit  vector  that  is  perpendicular  to  the  image- 
plane  . 

h,  v,  and  a  pass  through  F  and  are  the  image-plane  frame, 
j j_  is  the  projection  of  vector  j  on  axis  i. 

In  the  case  of  calibration  (and  to  make  the  calculations 
easier)  the  image-plane  frame  is  taken  to  be  at  (-|0F|)  on  the  y- 
axis  from  the  reference  frame.  The  orientation  of  both  frames 
being  the  same. 


Lffi.sfl.ting — a _ Zffiint _ of  interest 

First,  using  a  vision  processor,  the  location  of  the  image 
point  is  determined  with  respect  to  the  h.v  frame.  The  h  component 
is  "u"  and  the  v  component  is  'v'.  These  coordinates  are  then 
transformed  to  reference-frame  coordinates  by  applying  the 
following : 


Px  =  u.hx 
py  =  u.hy 
P2  =  u.h2 


+  v.vx  + 
+  v.vy  + 
+  V.v2  + 


f-ax  +  Fx 
f.ay  +  Fy 

f-az  +  Fz 


The  line  on  which  the  point  of  interest  lies  has  the 
following  parametric  equation: 

x  =  Fx  +  (px  -  Fx>  .t 

y  =  Fy  +  (Py  -  Fy) .t 

z  •  Fz  +  (Pz  -  Fz)  .t 

The  point  of  interest  "R"  is  located  on  the  intersection  of 
the  scanning-plane  with  the  above  line  (since  we  already  know  that 
the  point  of  interest  is  on  the  laser  line) .  Combining  the 
scanning-plane  equation  with  the  equation  of  the  above  line,  we 
get : 


_ _ (D  -  (a.Fx  +  B.Fy  4  C.Fz)) _ 

((A.px  +  B.py  +  C.pz)  -  (A.Fx  +  B.Fy  +  C.Fz)) 

Replacing  this  value  of  "t"  in  the  equations  for  the  line 
above,  we  get  the  coordinates  of  the  point  of  interest. 
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Examples 


The  following  coordinates  were  obtained  through  physical 
measurement : 

F (0, -41,0) 

B  (0,  0,  0) 

A(5 . 1, 0,  0) 

C  (5 . 1, 0,  -2 . 8) 

On  the  screen,  the  images  of  A,  B,  and  C  were  at  (84,0), 
(0, 0) ,  (84,-38) . 

From  the  above  data: 


f  A'B'  84 

f'BFlt-Ar=41x5T 


the  line  (A'F)  will  have  the  following  equation: 

x  =  -84  *t 

y  =  -41  -  675 . 2 94 *t 
2  =  0 

Knowing  that  C  is  on  the  plane  (y  =  0) ,  then  it  is  on  the 
intersection  of  that  plane  with  the  line  (A'F).  Applying  this  will 
give  us: 

t  =  -0.061 

then  the  coordinates  of  the  point  C  were  calculated  to  be : 

x  =  5.124 
y  =  0 


z  =  -2.318 

We  remark  here  a  discrepancy  between  the  calculated  and  the 
measured  values  for  the  coordinates  of  the  point  C.  This  is  due  to 
the  poor  measuring  instruments  that  were  used  during  the 
experiment .  Still  if  we  eliminate  all  these  errors,  there  will 
always  be  an  error  due  to  the  round-off  error  of  the  CPU  (which 
will  be  very  small)  and  the  resolution  of  the  camera-monitor 
system  ( which  will  also  be  small) . 

Another  set-up  was  made  and  this  time,  the  intersection  of 
the  line  with  the  scanning-plane  was  used  in  order  to  prove  the 
validity  of  the  algorithm  presented  above.  A  point  "S"  was  located 
on  the  laser  line  scanning  an  object.  The  plane  of  scanning  was 
determined  to  be: 

y  =  3.606 
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The  coordinates  of  that  point  on  the  screen  were  measured  to 


be : 


(52,  -33) 


The  camera-frame  was: 


h  =  -x 


v  =  -i 


m  =  y 

From  the  above  information,  the  poin.  under  consideration  was 
determined  to  be  on  the  line: 


x  =  -52*t 

y  -  -41  -  6  75 . 2  94  *t 
z  =  33*t 

The  intersection  of  that  line  with  the  plane  "y  =  3.606"  is: 


X  = 

3.43 

vs 

3.606 

y  = 

3.606' 

vs 

3.606 

z  = 

-2 . 178 

vs 

-2.800 

Another  source  of  error  in  the  calculation  above  is  the 
determination  of  the  plane  of  scanning  by  direct  measurement. 

The  VALii  Program 

A  VALii  program  was  written  to  perform  the  above.  The  listing 
of  that  program  follows: 


u  -  52 
v  «  -33 
xf  =  0 
yf  =  -41 
zf  =  0 

f  =  675.2942 
asp  =  0 
bsp  =  1 
csp  -  0 
dsp  =  3.606 
xh  =  1 
yh  =  0 
zh  =  0 
xv  *  0 
yv  =  0 
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ZV  —  1 

xa  =  0 
ya  =  1 
za  =  0 

xp  =  u.xh  +  v.xv  +  f . xa  +  xf 

yp  =  u . yh  +  v.yv  +  f.ya  +  yf 

zp  =  u.zh  +  v . zv  +  f.za  +  zf 

temp  =  asp.xf  +  bsp.yf  +  csp.zf 
tempi  =  asp.xp  +  bsp.yp  +  csp.zp 
t  =  (dsp  -  temp) /(tempi  -  temp) 


xr  = 

xf  +  (xp  - 

xf)  .t 

yr  = 

yf  +  (yp  - 

yf)  -t 

zr  = 

zf  +  (zp  - 

zf)  .t 

type 

/B,  x,  c2 

type 

/B,  y,  c2 

type 

/B,  z,  c2 

Future  Work 

Future  work  should: 

1.  Identify  and  implement  a  way  to  detect  a  laser  beam  on  the 
screen  (using  the  vision  processor) . 

2.  Identify  and  implement  a  way  to  detect  a  broken  line  (i.e. 
equations  of  the  straight  lines  constituting  a  broken  line) . 

3.  Further  research  towards  the  identification  and  nanaling  of 
non-rigid  3-D  objects . 
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II  .  Locating _ Surfaces _ Using  a  Range  Finder 


The  program  (listing  given  in  III)  analyzes  an  image  provided 
by  a  camera.  It  will  search  for  bright  lines  in  the  image  and 
determine  the  equations  and  boundaries  of  those  lines.  From  those 
boundaries,  the  boundaries  of  the  laser  lines  that  are  projected 
on  the  object  are  calculated  in  3-D  coordinates.  Thus  the 
equations  of  those  lines  are  calculated  and  the  surfaces  on  which 
they  are  projected  are  determined. 


Upper  Boundary  of  Line 


The  Line 


The  Screen 


Lower  Boundary  of  Line 


Boundaries  of  the  Image  of  a  Laser-Line 


The  program  begins  by  assuming  that  there  is  a  continuous 
(vertical)  laser-line  from  the  top  to  the  bottom  of  the  screen.  It 
tries  to  locate  the  boundaries  of  that  line.  Beginning  with  the 
top,  it  searches  for  a  bright  point.  If  that  point  is  found,  its 
2-D  position  is  saved;  otherwise,  the  search  is  restarted  at  the 
next  level  down  the  screen.  Once  the  upper  boundary  of  the  line  is 
located,  the  same  procedure  is  used  to  locate  the  lower  boundary 
(i.e.  search  for  a  bright  point  in  the  bottom  of  the  screen,  if 
not  found  then  move  to  the  next  level  up) .  Once  the  two  boundaries 
are  found,  the  equation  of  the  assumed  line  is  calculated.  The 
program  then  tries  to  locate  a  bright  point  midway  between  the  two 
levels . 

If  a  bright  point  is  located,  then  its  coordinates  are 
checked  whether  they  are  on  the  assumed  line  (or  near  it  by  some 
margin  of  error) .  If  this  is  the  case,  then  we  have  enough  reason 
to  believe  our  earlier  assumption.  If  the  point  is  not  on  or  near 
the  line,  then  the  program  assumes  that  there  are  two  distinct 
straight-lines  and  thus  updates  its  list  of  lines  to  reflect  the 
presence  of  both  lines  (instead  of  the  original  line) .  Then  the 
procedure  is  started  all  over  for  each  line  in  the  list. 

If  two  bright  points  were  located  (the  program  does  not 
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detect  more  than  two  points),  the  program  assumes  that  there  are 
two  distinct  straight-lines,  the  two  points  just  detected 
correspond  to  the  boundaries  of  those  lines  (the  beginning  of  the 
lower  line  and  the  end  of  the  higher  one) .  The  list  of  lines  to  be 
checked  is  updated  accordingly  and  the  procedure  is  repeated  for 
each  line  in  the  list . 

If  no  bright  point  has  been  detected,  then  there  are  two 
straight-lines,  the  upper  boundary  of  one  and  the  lower  boundary 
of  the  other  are  not  known.  The  program  locates  those  boundaries 
using  a  modification  of  the  procedure  that  located  the  boundaries 
of  the  initially  assumed  line.  After  locating  those  boundaries, 
the  list  of  lines  to  be  checked  is  updated  and  the  procedure  is 
repeated  for  each  line  on  the  list. 


Possible  Patterns  of  Lines  to  be  Detected 


Once  the  boundaries  of  a  straight-line  and  its  existence  are 
established,  the  program  returns  its  boundaries  and  moves  to  check 
the  next  line  on  the  list.  This  is  done  until  all  candidate  lines 
are  checked. 

The  output  of  this  program  can  be  used  by  our  earlier  program 
to  get  the  boundaries  of  the  laser-lines  in  3-D  coordinates. 
Knowing  those  boundaries,  we  can  derive  the  information  we  need 
about  the  object  (e.g.  depth  of  the  object  is  the  distance  between 
the  two  shifted  lines) . 

This  program  still  needs  some  refinements: 

1.  Experiment  with  different  objects  to  check  the  validity  of  the 
algorithm  described  above. 
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2.  Currently,  the  program  approximates  curves  by  a  series  of  two- 
point  lines.  This  matter  must  be  studied  further  to  determine  the 
best  way  to  identify  curves. 

3.  Many  assumptions  were  made  throughout  the  writing  of  this 
algorithm.  These  assumptions  are  valid  for  the  objects  that  were 
under  study  (a  cube  and  planar  surfaces),  the  validity  of  those 
assumptions  must  be  determined  for  a  larger  number  of  objects  (and 
if  possible  investigate  the  theory  behind  them) . 

4.  The  program  assumes  ideal  lines  in  the  image  (i.e.  width  of  the 
laser-line  is  1  pixel  on  the  screen) ,  a  way  to  identify  lines  with 
varying  width  should  be  investigated.  This  should  be  done  by 
experimenting  with  different  objects  to  identify  the  maximum  width 
of  a  laser  line  and  how  it  varies  with  the  type  of  material  of  the 
scanned  surface. 

5.  A  method  to  speed  up  the  search  for  lines  should  be 
investigated  if  the  current  method  does  not  meet  our  real-time 
constraints . 

Future  research  should  also  concentrate  on  object 
recognition.  A  method  should  be  found  to  recognize  rigid  as  well 
as  non-rigid  objects.  The  vision  and  range-finder  information 
integration  must  be  studied  as  well.  Also  another  area  of  study 
towards  the  recognition  of  non-rigid  objects,  is  the  database  of 
objects  that  should  be  built  and  the  Artificial  Intelligence 
aspect  behind  the  recognition  process. 
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111  •  The — Listing  of  the  Program 

The  program  consists  of  a  main  routine  (LASER. LINE)  and 
supporting  subroutines. 

The  listings  of  all  routines  follow: 

LASER . LINE 


Call  v.init 

Threshold  =  200 

Call  v. erase .graphics 

Call  v.scan 

Call  v. freeze 

Call  get. first 

If  level [1]  ==  255  goto  200 

k  -  0 

plist  *=  10 
nlines  =  1 

xbpoint [plist ]  =  u[l] 
ybpoint [plist ]  =  v[l] 
xepoint [plist ]  =  u[2] 
yepoint [plist ]  =  v[2] 

Call  LOCATE. LINE 
200  Type  /B, "No  More  Lines" 
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GET. FIRST 

level [1]  =  0 
level [2 ]  =  255 

1  j  =1 

lev  =  level [ 1 ] 

Call  DETECT. LIGHT 

if  (v. reply [1]  <  Threshold  and  level  [1]  <  255)  then 
level [1]  =  level [1]  +  1 
Cali  v . erase . graphics 
Goto  1 

End 

if  level [1]  ==  255  then  goto  100 
Call  GET. POINT 

2  j  =2 

lev  =  level [2] 

Call  DETECT. LIGHT 
if  (v.reply[l]  <  Threshold)  then 
level [2]  =  level [2]  -  1 
Call  v . erase . graphics 
Goto  2 

End 

Call  GET. POINT 
100  Return 
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LOCATE 


LINE 

3  u[l]  =  xbpoint [plist ] 
v[l]  =  ybpoint [plist ] 
u[2]  =  xepoint [plist] 
v[2]  =  yepoint [plist ] 
Call  BRROKEN . OR . NOT 
if  shift  then 


xepoint [plist  +  1]  = 
yepoint [plist  +  1)  = 
xbpoint [plist  +  1]  = 
ybpoint [plist  +  1]  = 
xepoint [plist ]  =  u[3] 
yepoint [plist ]  =  v[3] 
nlines  =  nlines  +  1 


xepoint [plist ] 
yepoint [plist] 
u  [4 ] 
v  [  4  ] 


End 


If 


connected  then 
if  broken  then 

xepoint  [plist  +  1]  = 
yepoint [plist  +  1]  = 
xbpoint [plist  +  1]  = 
ybpoint [plist  +  1]  = 
xepoint [plist ]  =  u[3] 
yepoint [plist ]  =  v[3] 
nlines  =  nlines  +  1 
Goto  3 


xepoint [plist ] 
yepoint  [plist ] 
u[3] 
v  [3] 


End 

xe[k]  *  xepoint [plist ] 

ye[k]  =  yepoint [plist ] 

xb[k]  =  xbpoint [plist ] 

yb[k]  =  ybpoint [plist ] 

plist  =  plist  +  1 

Type  /B,  "k  =  ",k 

Type  /B,  "xb  =  ", xb [k] , /X10, /S 

Type  /B,  "yb  =  ",yb[k] 

Type  /B,  "xe  =  ", xe [k] , /X10, /S 
Type  /B,  "ye  =  ",ye[k] 
k  «  k  +  1 


nlines  =  nlines  -1 
if  nlines  goto  3 
Return 


Else 


Call  LOCATE. DISCONNECTION 
nlines  =  nlines  +  1 
Goto  3 

End 


58 


59 


BROKEN . OR . NOT 


a  =  (v[l]  -  v  [2  ]  )  /  (u  [  1  ]  -  u  f  2  ]  ) 
b  =  v[l]  -  a*u  [1] 
level [3]  =  128  -  <v[2]  +  v[l])/2 
lev  =  level [3] 

Call  DETECT. LIGHT 
if  V. Reply [1]  <  Threshold  then 
connected  =  0 

else 

connected  =  1 
Call  GET. POINT 

if  abs (v [ 3 ]  -  t*u[3]  -  b)  >  2  then 
broken  =  1 


else 


Return 


DETECT. LIGHT 


v.x  =  0 
v.y  =  lev 
Call  v.move .pointl 
v.x  =  255 

Call  v.move .point2 
Call  v. draw. line 
v. spacing  =  0 
Call  v.get.minmax 
Return 
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GET. POINT 


one  =  0 

Call  v. get. graph 

For  i  =  1  to  255 

if  v.reply[i]  >  Threshold  then 
if  one  then  j  =  4  End 
v[j]  =  128  -  v.y 
u[ j]  =  i  -  128 
one  =  1 

End 

if  one  then  shift  End 
one  =  0 

if  u  [4 ]  <  u[3]  then 
temp  =  v[3] 
v  [3  ]  =  v  [  4  ] 
v[4]  =  temp 
temp  =  u[3] 
u  [3]  =  u [4] 
u[4]  =  temp 

End 

End 

Return 
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LOCATE .DISCONNECTION 


lev  =  level  [3] 

4  lev  *=  lev  -  1 
Call  DETECT. LIGHT 

if  V. Reply [1]  <  Threshold  then 
goto  4 

else 

j  =  3 

Call  GET. POINT 
xepoint [plist ]  =  u[3] 
yepoint [plist ]  =  v[3] 

End 

lev  =  level [3] 

5  lev  *  lev  +  1 
Call  DETECT. LIGHT 

if  V. Reply [1]  <  Threshold  then 
goto  5 

else 

j  =  3 

Call  GET. POINT 
xbpoint [plist ]  =  u[3] 
ybpoint [plist ]  =  v[3] 

End 

Return 


